Large language models' (LLMs) abilities are drawn from their pretraining data, and model development begins with data curation. However, decisions around what data is retained or removed during this initial stage is under-scrutinized. In our work, we ground web text, which is a popular pretraining data source, to its social and geographic contexts. We create a new dataset of 10.3 million self-descriptions of website creators, and extract information about who they are and where they are from: their topical interests, social roles, and geographic affiliations. Then, we conduct the first study investigating how ten "quality" and English language identification (langID) filters affect webpages that vary along these social dimensions. Our experiments illuminate a range of implicit preferences in data curation: we show that some quality classifiers act like topical domain filters, and langID can overlook English content from some regions of the world. Overall, we hope that our work will encourage a new line of research on pretraining data curation practices and its social implications.

When Ross Compton had a pacemaker installed, he had a constitutional right to remain silent. One would expect his body to have the same. But when the 59-year-old's Middletown, Ohio, home erupted in flames last September, the electronic data stored in his cardiac device eventually led to his arrest and subsequent indictment on charges of arson and insurance fraud. And despite his attorney's arguments to the contrary, earlier this month Butler county judge Charles Pater held that the functioning of Compton's own body -- heartbeat included -- could be used against him at the upcoming trial. Deanna Paul (@thedeannapaul) is a former New York City prosecutor and adjunct professor of trial advocacy at Fordham University school of law.